1. Field of the Invention
This invention relates to machine vision systems and more particularly to uses for advanced machine vision search tools that register patterns transformed by multiple degrees of freedom.
2. Background Information
The use of advanced machine vision systems and their underlying software is increasingly employed in a variety of manufacturing and quality control processes. Machine vision enables quicker, more accurate and repeatable results to be obtained in the production of both mass-produced and custom products. Basic machine vision systems include one or more cameras (typically having solid-state charge couple device (CCD) is imaging elements) directed at an area of interest, frame grabber/image processing elements that capture and transmit CCD images, a computer and display for running the machine vision software application and manipulating the captured images, and appropriate illumination on the area of interest.
Many applications of machine vision involve the inspection of components and surfaces for defects that affect quality. Where sufficiently serious defects are noted, a part of the surface is marked as unacceptable/defective. Machine vision has also been employed in varying degrees to assist in manipulating manufacturing engines in the performance of specific tasks. One task using machine vision is visual servoing of robots in which a robot end effector is guided to a target using a machine vision feedback. Other applications also employ machine vision to locate a stationary and/or moving pattern.
The advent of increasingly faster and higher-performance computers, has enabled the development of machine vision systems that employ powerful search tools. Such search tools enable a previously trained/stored image pattern to be acquired and registered/identified regardless of its viewed position. In particular, existing commercially available search tools can register such patterns transformed by at least three degrees of freedom, including two translational degrees (x and y-axis image plane) and a non-translational degree (rotation and/or scale, for example). One particular implementation of an advanced search tool is the rotation/scale-invariant search (RSIS) tool. This tool registers an image transformed by at least four degrees of freedom including the two translational degrees (x and y-axis image plane) and at least two non-translational degrees (z-axis(scale) and rotation within the x-y plane about an axis perpendicular to the plane). Some tools also register more complex transformations such as aspect ratio (rotation out of the plane whereby size on one axis decreases while size in the transverse axis thereto remains the same). These search tools, therefore, enable a specific pattern within the field of view to be located within a camera field of view to be positively identified and located accurately within the vision system""s internal reference system (an x, y, z, rotation coordinate system, for example). The RSIS and other advanced search tools particularly allow for the identification and acquisition of patterns having somewhat arbitrary rotation, scaling (e.g. distancing) and translation with respect to the reference system. In other words, the tool is sufficiently robust to recognize a desired pattern even if it is rotated and larger/smaller/skewed relative to a xe2x80x9cmodelxe2x80x9d or trained pattern within the vision system.
In general, advanced machine vision tools acquire an image of a pattern via a camera and analyze the outline or a particular part of the pattern, such as a predetermined fiducial mark. The processing speed of the underlying computer in which the tool resides is sufficient to enable a very large number of real time calculations to be completed in a short time frame. This particularly enables the search tool to determine the coordinates within an image reference system for each analyzed point in the viewed area, and correlate these through repetition with a desired pattern. The search tool may map the locations of various points in the captured image to stored points in the model image, and determine whether the captured image points fall within an acceptable range of values relative to the model image points. Using various decision algorithms, the tool decides whether the viewed pattern, in a particular rotation and distance (scale) corresponds to the desired search pattern. If so, the tool confirms that the viewed pattern is, in fact, the pattern for which the tool is searching and fixes its position and orientation.
Machine vision systems having a four-degree-of-freedom, or greater, capability (such as RSIS) are available from a number of commercial vendors including Hexavision(copyright) from Adept Technology, Inc. of San Jose, Calif., and the popular Patmax(copyright) system from Cognex Corporation of Natick, Mass. Advanced machine vision search tools such as Patmax(copyright) also have the ability to take advantage of the previous known position of a search subject or target. This narrows the search area to positions relatively near the last known location. Therefore, searching is relatively faster on the next cycle since a smaller area is searched. In addition, these search tools can tolerate partial occlusion of a pattern and changes in its illumination, adding further to their robustness with respect to less advanced machine vision approaches.
In general, when a camera views an object, it resolves an imaged pattern on the object into an image plane that, as defined herein, is represented by the x and y axes of a three-dimensional coordinate system. These are two translational axes in which the camera can register transformation of the pattern directly based upon observed position within the overall camera field of view. In addition, the camera axis perpendicular to the image plane can be represented as the z-axis, which, as noted is generally represented as a non-translational scale measurement (the larger the pattern, the closer it is to the camera and vice versa). This axes can also be measured by a special ranging camera. The three orthogonal axes (x, y and z) define three degrees of freedom with respect to the viewed object. In addition rotation of the viewed pattern of the object about three axes (typically characterized as roll, pitch and yaw rotations ("psgr", xcfx86, xcex8) about the respective (x, y, z) axes) can also be present with respect to the image plane. These rotations account for three additional degrees of freedom. When an object is rotated by xcex8, it appears, likewise rotated about the z/camera axis in the image plane with no change in width-along the image-plane other axes. When it is rotated about only one of either "psgr" or xcfx86, it appears to have a changed aspect ratio (i.e. narrowed along one image plane axis as it rotates about the opposing image plane axis. Note that this form of rotation actually changes the viewed outline of the pattern with respect to the search tool""s reference frame (i.e. a circle becomes an oval). If rotation about both image-plane axes occurs, then the pattern shows a shear. Using a four-degree-of-freedom search tool, it can be difficult to accurately register and locate a trained pattern that exhibits transformation along all six degrees of freedom including aspect and shear with respect to the image plane, as the trained pattern, itself undergoes change to its overall shape and size in a manner that may not be easily predicted or recognized by a search tool.
Accordingly, it is an object of this invention to provide a system and method for measuring patterns transformed by six degrees of freedom using a machine vision search tool having, generally the ability to register patterns transformed by four degrees of freedom. The system and method should enable training and registration of a pattern particularly based upon transformations along degrees of freedom that are not readily accommodated by the underlying machine vision search tool including aspect and shear.
This invention overcomes the disadvantages of the prior art by providing a system and method for utilizing a search tool that finds and/or registers (finds and locates) transformation of a trained pattern by at least four degrees of freedom to register the instance of a pattern in an arbitrary six-degree-of-freedom pose.
According to a preferred embodiment the system is first trained by providing multiple patterns corresponding to different aspect and shear (pan and tilt) with respect to an original template pattern. In one embodiment, training can involve a synthetic (or physical) panning and tilting of the template pattern to generate the desired plurality of instances of the trained pattern at different amounts of aspect/shear aspect (e.g. the two degrees of freedom not handled by a four-degree-of freedom search tool). In an alternate embodiment, in cases where the search tool allows a user to specify differing levels of aspect/shear with respect to an image, training can entail the storage of a particular template pattern that is subsequently transformed based upon the user-specified aspect/shear.
Once the search tool has been trained with the appropriate training patterns, the search tool is directed to acquire an image of an object containing one or more instance of the trained pattern(s). In a preferred embodiment, the instances of the trained pattern are found, and located (e.g. registered) so that the six-degree-of-freedom pose of the underlying object can be determined.
According to one embodiment, a single instance of a fiducial or trained pattern on the object can be imaged and registered. The search tool can be run using the image with each of a plurality of trained patterns, each trained pattern representing a differing aspect/shear. The result from each run can be scored, and the score that is highest can be identified as the prevailing aspect and shear associated with the object. Score can be determined based upon interpolation, using a parabolic fit of a given highest-scoring fiducial versus close neighbors on a parabola. Alternatively, a gradient descent can be used to determine the closest-matching training fiducial in terms of aspect and shear to the runtime image fiducial.
According to another embodiment multiple fiducials or subpatterns of a fiducial on the object are located based upon the trained pose information therefor and the relative six-degree-of-freedom pose is measured by providing a transformation between the located positions for other fiducials/subpatterns with respect to the base fiducial/subpattern and the expected positions for other fiducials/subpatterns with respect to a base fiducial/subpattern. The resulting position/orientation of the object is then determined.
It is contemplated that the fiducials/subpatterns according to this invention can be found only, (e.g. not also located/registered) according to an alternate embodiment, using the techniques described herein.